…4941)

* Fixing populateMeta fields update to metadata table

* Fix checkstyle violations

Co-authored-by: Sagar Sumit <sagarsumit09@gmail.com>

… interfaces first (#4942)

* In some complex network environment, the current code returns wildcard address 0.0.0.0 which is not desired.

…nc.enable" directly, async clustering not work (#4905)


Co-authored-by: Rex An <bonean131@gmail.com>

…d for Spark SQL (#4901)

* [HUDI-3445] Clustering Command Based on Call Procedure Command for Spark SQL

* [HUDI-3445] Clustering Command Based on Call Procedure Command for Spark SQL

* [HUDI-3445] Clustering Command Based on Call Procedure Command for Spark SQL

Co-authored-by: shibei <huberylee.li@alibaba-inc.com>

* flink TM memory Optimization

* Fixing timeline server for repeated refreshes

…partitions having different schemas (#4468)

* Fixing Hive getSchema for RT tables

* Addressing feedback

* temp diff

* fixing tests after spark datasource read support for metadata table is merged to master

* Adding multi-partition schema evolution tests to HoodieRealTimeRecordReader

Co-authored-by: Aditya Tiwari <aditya.tiwari@flipkart.com>
Co-authored-by: sivabalan <n.siva.b@gmail.com>

…ing _hoodie_is_deleted column to schema (#4921)

…cess before it is transformed to DataSet (#4930)

…ssary hoodie records (#4932)

* log scanner optimization
* payload equals switches to `=`

Co-authored-by: 苏承祥 <sucx@tuya.com>

…4811)

* Making commit preserve metadata to true

* Fixing integ tests

* Fixing preserve commit metadata for metadata table

* fixed bootstrap tests

* temp diff

* Fixing merge handle

* renaming fallback record

* fixing build issue

* Fixing test failures

… object into the closure for Spark to serialize (#4954)

- Avoid including whole MultipleSparkJobExecutionStrategy object into the closure for Spark to serialize

…y on HDFS (#4739)

- This change makes sure MT records are updated appropriately on HDFS: previously after Log File append operations MT records were updated w/ just the size of the deltas being appended to the original files, which have been found to be the cause of issues in case of Rollbacks that were instead updating MT with records bearing the full file-size.

- To make sure that we hedge against similar issues going f/w, this PR alleviates this discrepancy and streamlines the flow of MT table always ingesting records bearing full file-sizes.

Co-authored-by: yuezhang <yuezhang@freewheel.tv>

)

For flink insert overwrite operation, do the cleaning each time before the write.

close #4959

…#4970)

…lter construction from index based on the type param (#4848)

Rework of #4761 
This diff introduces following changes:

- Write stats are converted to metadata index records during the commit. Making them use the HoodieData type so that the record generation scales up with needs. 
- Metadata index init support for bloom filter and column stats partitions.
- When building the BloomFilter from the index records, using the type param stored in the payload instead of hardcoded type.
- Delta writes can change column ranges and the column stats index need to be properly updated with new ranges to be consistent with the table dataset. This fix add column stats index update support for the delta writes.

Co-authored-by: Manoj Govindassamy <manoj.govindassamy@gmail.com>

…ulti processors at once (#4968)

Desc: Add a hive sync config(hoodie.datasource.hive_sync.sync_comment). This config defaults to false.
While syncing data source to hudi, add column comments to source avro schema, and the sync_comment is true, syncing column comments to the hive table.

…ng buildx (#5011)

…olumns from schema (#4972)

* [HUDI-3522] Introduce DropColumnSchemaPostProcessor to support drop columns from schema

* Fix case sensitivity

* [HUDI-2999] rfc for consistent hashing index

* [HUDI-2999] review: add metadata table & non-dual-write solution (virtual log file) for resizing

Co-authored-by: xiaoyuwei <xiaoyuwei.yw@alibaba-inc.com>

Co-authored-by: 苏承祥 <sucx@tuya.com>

… in TestSchemaPostProcessor (#5019)

…4982)

…load previous Index Table state (#5015)

#4999)

Co-authored-by: Rex An <bonean131@gmail.com>

…etting multi processors at once (#4969)

…d compaction if rollback failed mid-way (#4971)

#5013)

Create new TypedProperties while performing clustering

Add OrderedProperties and minor refactoring

Add javadoc and remove getters from OrderedProperties

…4984)

Co-authored-by: Y Ethan Guo <ethan.guoyihua@gmail.com>

…5025)